Field of Invention
Aspects of this invention are related to control of minimally invasive surgical systems, and are more particularly related to using hand movement of a surgeon in controlling a minimally invasive surgical system.
Related Art
Method and techniques for tracking hand positions and gestures are known. For example, some video game controllers utilize hand tracking input. For example, the Nintendo Wii® gaming platform supports wireless position and orientation sensing remote controls. (Wii® is a U.S. registered trademark of Nintendo of America Inc., Redmond Wash., U.S.A.) The use of gestures and other physical motions like swinging a bat or waving a magic wand provide the fundamental gaming element for this platform. The Sony Playstation Move® gaming platform has features similar to those of the Nintendo Wii® gaming platform.
A wireless CyberGlove® motion capture data glove from CyberGlove Systems includes eighteen data sensors with two bend sensors on each finger, four abduction sensors and sensors measuring thumb crossover, palm arch, wrist flexion, and wrist abduction. (CyberGlove® is a registered trademark of CyberGlove Systems LLC of San Jose, Calif.) When a three-dimensional tracking system is used with the CyberGlove® motion capture data glove, x, y, z, yaw, pitch, roll position, and orientation information for the hand are available. The motion capture system for the CyberGlove® motion capture data glove has been used in digital prototype evaluation, virtual reality biomechanics, and animation.
Another data glove with forty sensors is the ShapeHand data glove of Measurand Inc. A ShapeClaw portable, lightweight hand motion capture system of Measurand Inc. includes a system of flexible ribbons that capture index finger and thumb motion along with position and orientation of the hand and forearm in space.
In In-Cheol Kim and Sung-Il Chien, “Analysis of 3D Hand Trajectory Gestures Using Stroke-Based Composite Hidden Markov Models”, Applied Intelligence, Vol. 15 No. 2, p. 131-143, September-October 2001, Kim and Chien explore the use of three-dimensional trajectory input with a Polhemus sensor for gesture recognition. Kim and Chien propose this form of input because three-dimensional trajectories offer more discriminating power than two-dimensional gestures, which are predominantly used in video-based approaches. For their experiments, Kim and Chien made use of a Polhemus magnetic position tracking sensor attached to the back of a Fakespace PinchGlove. The PinchGlove provides a means for the user to signal the beginning and ending of a gesture while the Polhemus sensor captures the three-dimensional trajectory of the user's hand.
In Elena Sanchez-Nielsen, et al., “Hand Gesture Recognition for Human-Machine Interaction,” Journal of WSCG, Vol. 12, No. 1-3, ISSN 1213-6972, WSCG '2004, Feb. 2-6, 2003, Plzen Czech Republic, a real time vision system is proposed for application within visual interaction environments through hand gesture recognition using general purpose hardware and low cost sensors, like a personal computer and a web cam. In Pragati Garg, et al., “Vision Based Hand Gesture Recognition,” 49 World Academy of Science, Engineering and Technology, 972-977 (2009), a review of vision-based hand gesture recognition was presented. One conclusion presented was that most approaches rely on several underlying assumptions that may be suitable in a controlled lab setting but do not generalize to arbitrary settings. The authors stated “Computer Vision methods for hand gesture interfaces must surpass current performance in terms of robustness and speed to achieve interactivity and usability.” In the medical area, gesture recognition has been considered for sterile browsing of radiology images. See Juan P. Wachs, et al., “A Gesture-based Tool for Sterile Browsing of Radiology Images”, Journal of the American Medical Informatics Association (2008; 15:321-323, DOI 10.1197/jamia.M24).